A general “power-of-d” dispatching framework for heterogeneous systems

Intelligent dispatching is crucial to obtaining low response times in large-scale systems. One common scalable paradigm the “power-of-d,” which dispatcher queries d servers at random and assigns job a server based only on state of queried servers. The bulk power-of-d policies studied literature assume that system homogeneous, meaning all have same speed; meanwhile, real-world systems often exhibit speed heterogeneity. This paper introduces general framework for describing analyzing heterogeneity-aware policies. key idea behind our can make use information two decision points: when choosing query assigning one those Our explicitly separates policy into querying rule an assignment rule; we consider families both types. While strongest rules incorporate detailed queue-length information, these typically are difficult analyze. We overcome this difficulty by focusing ignore queue length beyond idleness status. In setting, analyze mean time formulate novel optimization problems joint assignment. build upon optimized develop heuristic length-aware perform well simulation, relative appeared literature.